The present invention relates to the field of media processing and, more particularly, to system and methodology for efficient transmission of media content (e.g., digital images, sound, and/or video) from wireless devices (e.g., digital cameras with wireless capability or connectivity to cellular phone devices).
Today, digital imaging, particularly in the form of digital cameras, is a prevalent reality that affords a new way to capture photos using a solid-state image sensor instead of traditional film. A digital camera functions by recording incoming light on some sort of sensing mechanisms and then processes that information (basically, through analog-to-digital conversion) to create a memory image of the target picture. A digital camera's biggest advantage is that it creates images digitally thus making it easy to transfer images between all kinds of devices and applications. For instance, one can easily insert digital images into word processing documents, send them by e-mail to friends, or post them on a Web site where anyone in the world can see them. Additionally, one can use photo-editing software to manipulate digital images to improve or alter them. For example, one can crop them, remove red-eye, change colors or contrast, and even add and delete elements. Digital cameras also provide immediate access to one's images, thus avoiding the hassle and delay of film processing. All told, digital photography is becoming increasingly popular because of the flexibility it gives the user when he or she wants to use or distribute an image.
The defining difference between digital cameras and those of the film variety is the medium used to record the image. While a conventional camera uses film, digital cameras use an array of digital image sensors. When the shutter opens, rather than exposing film, the digital camera collects light on an image sensor, a solid state electronic device. The image sensor contains a grid of tiny photosites that convert light shining on them to electrical charges. The image sensor may be of the charged-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) varieties. Most digital cameras employ charge-coupled device (CCD) image sensors, but newer cameras are using image sensors of the complimentary metal-oxide semiconductor (CMOS) variety. Also referred to by the acronym CIS (for CMOS image sensors), this newer type of sensor is less expensive than its CCD counterpart and requires less power.
During camera operation, an image is focused through the camera lens so that it will fall on the image sensor. Depending on a given image, varying amounts of light hit each photosite, resulting in varying amounts of electrical charge at the photosites. These charges can then be measured and converted into digital information that indicates how much light hit each site which, in turn, can be used to recreate the image. When the exposure is completed, the sensor is much like a checkerboard, with different numbers of checkers (electrons) piled on each square (photosite). When the image is read off of the sensor, the stored electrons are converted to a series of analog charges which are then converted to digital values by an Analog-to-Digital (A to D) converter, which indicates how much light hit each site which, in turn, can be used to recreate the image.
In order to generate an image of quality that is roughly comparable to a conventional photograph, a substantial amount of information must be capture and processed. For example, a low-resolution 640×480 image has 307,200 pixels. If each pixel uses 24 bits (3 bytes) for true color, a single image takes up about a megabyte of storage space. As the resolution increases, so does the image's file size. At a resolution of 1024×768, each 24-bit picture takes up 2.5 megabytes. Because of the large size of this information, digital cameras usually do not store a picture in its raw digital format but, instead, apply compression technique to the image so that it can be stored in a standard-compressed image format, such as JPEG (Joint Photographic Experts Group). Compressing images allows the user to save more images on the camera's “digital film,” such as flash memory (available in a variety of specific formats) or other facsimile of film. It also allows the user to download and display those images more quickly.
During compression, data that is duplicated or which have no value is eliminated or saved in a shorter form, greatly reducing a file's size. When the image is then edited or displayed, the compression process is reversed. In digital photography, two forms of compression are used: lossless and lossy. In lossless compression (also called reversible compression), reversing the compression process produces an image having a quality that matches the original source. Although lossless compression sounds ideal, it does not provide much compression. Generally, compressed files are still a third the size of the original file, not small enough to make much difference in most situations. For this reason, lossless compression is used mainly where detail is extremely important as in x-rays and satellite imagery. A leading lossless compression scheme is LZW (Lempel-Ziv-Welch). This is used in GIF and TIFF files and achieves compression ratios of 50 to 90%.
As shown in FIG. 1A, the system 100 includes a Sensor 101, a Shutter Actuator 103, an Image Processor 102, an Image (DRAM) Memory 104, a (Central) Processor 106, a Keypad and Controls 108, a Program Code Flash Memory 107, a (System) Memory 105, a Direct View Display or Viewfinder 109, a Hot Shoe Interface 110, and a “Digital Film” Flash Memory 111. As illustrated, these various components communicate with one another using a bus architecture including, for instance, an Address Bus, a Data Bus, and an I/O (Input/Output) Bus.
The system 100 employs the Sensor 101 for basic image capture. The Sensor 101 operates, in essence, by capturing light and transforming that into electrical voltage levels. A suitable sensor is available from a variety of vendors, including VLSI Vision, Motorola, and Toshiba. In a preferred embodiment, the Sensor 101 includes, for example, a 1280×1024 color CMOS sensor, such as a VLSI Vision VVL 6801 CMOS sensor. However, other sensor technology is suitable, including CCD sensors.
The Sensor 101 must, of course, be part of a larger assembly to operate. Specifically, the Sensor 101 operates in conjunction with a lens assembly (not shown), or other optics to focus an image onto the sensor. The optics themselves are controllable, for instance, using a conventional aperture, focus, and shutter control mechanisms. The currently-preferred embodiment uses an 18 mm fixed-focal length, fixed-aperture lens assembly to provide a broad depth of field. The lens assembly employs two manual slide controls, a macro lens control, and an exposure control. The macro lens control switches from normal to close-up mode by sliding a macro lens in and out of the lens assembly to provide normal or extreme close-up capability. The exposure control switches from normal to bright light by sliding a neutral gray filter in and out of the lens assembly. Aside from choosing normal or bright light, normal or close-up mode, the camera requires no manual focusing, shutter speed or aperture adjustment. Operation is as simple as point and shoot. The Sensor 101, on the other hand, operates under the control of the Image Processor 102, which will now be described.
The Image Processor 102, which basically operates as a state machine, provides overall control for the Sensor 101. In operation, the Image Processor 102 controls
Although it is possible to compress images without losing some quality, it is not practical in many cases. Therefore, all popular digital cameras use a lossy compression. Although lossy compression does not uncompress images to the same quality as the original source, the image remains visually lossless and appears normal. In many situations, such as posting images on the Web, the image degradation is not obvious. The trick is to remove data that is not obvious to the viewer. For example, if large areas of the sky are the same shade of blue, only the value for one pixel needs to be saved along with the locations of where the other identical pixels appear in the image.
The leading lossy compression scheme is JPEG (Joint Photographic Experts Group) used in JFIF files (JPEG File Interchange Format). JPEG is a lossy compression algorithm that works by converting the spatial image representation into a frequency map. A Discrete Cosine Transform (DCT) separates the high- and low-frequency information present in the image. The high frequency information is then selectively discarded, depending on the quality setting. The greater the compression, the greater the degree of information loss. The scheme allows the user to select the degree of compression, with compression ratios between 10:1 and 40:1 being common. Because lossy compression affects the image, most cameras allow the user to choose between different levels of compression. This allows the user to choose between lower compression and higher image quality or greater compression and poorer image quality.
Today, all sorts of different types of information content may be captured digitally by various recording or capturing devices. In addition to digital photographic images, other examples of media include digital video and digital audio. Typically, once information content is captured, it is then transmitted or “uploaded”—either using wireless or wireline transmission means—to another host device, such as a server computer. Here, a problem exists as to how one transfers information content in a reliable, secure manner. For instance, in the case of a portable digital camera device, how can the user transmit captured digital images to a Web-based server computer in a reliable, secure manner, particularly if a wireless communication medium is employed. Or in the case of the digital audio, how does the user transmit dictation captured on a wireless handheld device to a remote host device. Of course the process is not merely limited to transmitting information content from a recording device to a host device. Instead, the process can be reversed such that information content is transmitted from a host device to a client device that is capable of displaying or rendering that information content. For example, a user may download e-mail information for displaying at a wireless handheld device. Regardless of whether information is uploaded or downloaded, the problem still remains, particularly when using wireless transmission means, as to how one can transmit information content in a reliable, secure manner.
A particular problem is encountered with today's wireless networks. Currently, wireless networks employ TCP/IP over communication networks supporting a transmission rate of only 9600 baud. This yields an effective throughput of only about 1 K (kilobytes) per minute. At such poor effective throughput rates, existing wireless networks pose a substantial bottleneck for the transmission of digital information content. For instance, a digital photographic image of 600 K may require several minutes to transmit. A relatively small MP3 audio file, say at 2 M (megabytes), would expectedly take much longer. A modest digital video file might even require several hours to transmit over such a low-bandwidth communication link.
However, the problem is even worse than the foregoing suggests. Because of limitations today in existing wireless systems, often a “call” (i.e., session) will be lost (i.e., prematurely terminated). Practically all cellular phone users have experienced firsthand the annoying experience of having a cellular phone call dropped. By the very nature that wireless systems are mobile, wireless calls are often dropped due to natural or physical obstructions, such as interference from mountains. At other times, however, a call may be dropped simply due to unreliability in the underlying cellular phone network.
Regardless of the cause of a user's call having been lost, the user is often forced to not only re-establish communication but also resend the entire contents of the information (e.g., entire digital photo) that was previously being transmitted, since most systems treat information content on a per-item basis. If a wireless call is lost during transmission of a digital image, for instance, those systems require retransmission of the entire digital image, as each image is treated as a single unit. There is no capability to benefit from the previous transmission of a portion of that digital image.
This retransmission problem is by no means limited to wireless systems. For example, when uploading a photographic image using a 56 K modem, any loss of the connection will result in the user having to completely retransmit the photo, despite the fact that the user may have previously uploaded a substantial portion of that very same photo. Moreover, the comparatively slower transmission rates available in wireless systems certainly can be expected to exacerbate the problem.
One attempt to address these problems, at least in the wireless arena, is to reinvent the communication protocols employed. Instead of using TCP/IP, this approach would employ, instead, a proprietary protocol that attempts to upload information content in pieces (i.e., not all-or-none). This would address, for instance, the above-mentioned problem of an aborted transmission, by allowing a system to resume transmission where transmission was left off at (i.e., without retransmitting pieces that have already been successfully transmitted). Additionally, this has the benefit of adopting a different packet format, thereby decreasing the high overhead incurred with using TCP/IP packet format.
However, apart from the attempt to address aborted transmissions, there has been little or no effort to date to provide a comprehensive solution to the problem of low effective throughput that is offered by existing wireless systems—a problem which results not only from the requirement of retransmission but also from the low baud rate and frequent interruptions in service. Thus, even though such an approach attempts to address the problem of retransmission, the more fundamental problem of losing connections remains wholly unaddressed. All told, even when employing alternative communication protocols, gains in wireless throughput are modest, at best. A better solution is sought.